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STAP OPTIONS FOR SAFE 


1 . Introduction 


On 18 March 1980 we forwarded to you eight questions 
regarding the future evolution of the SAFE system and the 
relation of CIA SAFE to other community systems (See 
Appendix I), in this report we propose various actions 
which, . 1 f implemented, could yield a more valuable community 
asset in the long run. Because of the size of the system and 
its complexity, a delay (6 months to two years) in IOC can 
be anticipated. Productive use of this delay time can be 
made, as we discuss in subsequent sections. 

In our examination of SAFE, we were impressed with the 
need for a community manager of the ADP-communi cat ion 
systems. None now exists and SAFE is not being integrated 
into an overall community architecture. As a result the 
incremental value of SAFE will be less than it could be. 
r ,^, T ^ wlt!h °ut a community manager, the future capabilities of 
oABL could be strengthened and possible steps in that 
direction are described in Section 2. The longer term 
questions of technical direction of the overall Intelligence 
Community ADP-communi cat i on systems will be the subject of 
additional STAP analysis and should be considered a separate 
issue from that of SAFE. 


The evolutionary development of SAFE will require 
analysis of how th e communi ty uses SAFE. Sections 3 and 4 
describe means by which such analysis could proceed. As SAFE 
comeo on line, it will be essential for future planning to 
evaluate its usefulness. A possible means for evaluation is 
described in Section 5. 

Finally, we believe that a rich body of experience in 
systems similar to SAFE exists and could be beneficially 
applied to enhancing SAFE'S capabilities. To this end, we 
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propose in Section 6 establishment of an Advisory Council on 
Technology for SAFE. 

This report primarily examines CIA SAFE and its use by 
the Intelligence Community. Our emphasis on CIA SAFE derives 
from the fact that NFAC stands to benefit greatly from a 
truly operational SAFE. DIA needs center on the accuracy, 
maintenance capability and general utility of their large 
encyclopedic files. These files require restructuring and 
improved maintenance capability as well as a high level of 
concurrency in use. Our analysis focuses on the analyst 
support function of SAFE; this function is of secondary 
importance to DIA. DIA's requirement can more easily be met 
than CIA's. Thus while we propose a "true pilot SAFE" for 
CIA, such an activity should not impede the development of 
the DIA SAFE. Indeed, the. lessons learned in the "true pilot 
SAFE" will be of use in the evolutionary development of the 
DIA system. 

In outlining the options for the future, we are fully 
aware of and appreciate the concerns of the SAFE management 
office and the Office of Central Reference. The steps 
outlined below will delay the schedu led delivery of an 
operational system, but we believe that the present schedule 
cannot be met since such critical items as command language 
and central hardware have not yet been decided upon. The 
delay we anticipate can be put to use to obtain operational 
experience on a "true pilot SAFE." The delays whether 
anticipated or not will cause problems with 0MB and Congress 
and these should be recognized now. 


2. Strengthening the SAFE Management 


Strengthening of the community management of SAFE is 
essential if it is to become effective in satisfying its 
prescribed functions, and be capable of expanding flexibly 
and responsibly to aid the entire community. 
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mar. ailect the future performance of a large part of fhe 
community not be made by default by those 
e . xer ?^ se tIlG corresponding responsibility, 
should not be delegated either 
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who do not 
Such decision: 


to lower echelons 


or 


to 


The 

effort. 


context of SAFE 


To .,u . . . . ls a large and intimidating R&D 
Technical decisions must be 

aggressive contractor actions and critical nnoninrt 
operational tasks. These decisions may commit the J 
far beyond the decisionmaker's ken. 


made amidst both 


communi ty 


Management itself, 
to deploy experience and 
least: 


as an abstract entity, must be able 
judgment in the following areas, at 


1) Intelligence Production 

a) Collection 

b) Processing 

c) Storage and retrieval 

d) Analysis 

e) Output and distribution 


2) Analytic processes: 
substantive areas of intell 
political, military, or S&T. 


Some experience 
igence ; that 


and grasp in 
is, economic, 


3) Technology and R&D for Intelligence 

a) Information sciences 

b) Communications 

c) Computer technology 

4) Management and Human Factors 


Whatever the particular details of the management 

capacity 6 ' k that ex P erience and decisionmaking 

capacity must be accompanied by the annronriat-P 

authority— that requires rank and status to match the 
control to be exercised. 


Given the ongoing function 
Technology and R&D (number 3) is 
fill, because it has been 


of intelligence production, 
the most difficult area to 
and is changing so fast. 
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task De but at the overin yS neCeSSary in ai ^ large management 
delegated t 11 management responsibility cannot be 

technological' l {*! managers cannot themselves be 
technologically knowledgeable enough to monitor the 

n ° Zb? s e = 1Si0n f ab ° Ut d6Slgn «* per form. nee, tKey 
must establish a mechanism to ensure that decisions are 
adequately monitored and verified. — 

There are several possible ways of doing that: 

d) Hire, beg, or borrow deputies with the needed 
competencies. 


2) Borrow 
operating offices 
is t h a t loyalties 
at best divided. 


consulting as needed from other 
in the community. A problem with this 
and motivation will almost surely be 


3) Establish 
Technology (ACT) 


1 continuing Advisory Council on 
experts. This i s' ’ usluf" ‘ i/anco T’Hl 
and'performancef eC1Sl0nS/ ^ f ° l " monitori ^ Erections 


4 ) E; 
impo rtant 


;tablish similar 
decisions. Such 


Ad Hoc Panels for particular 

. . panels must invest a lot of 

time becoming aware of the context of the problem, so 
their cost-effectiveness will be low on the 
Furthermore, they cannot perform the 
function. They may be essential 
contingencies. 


in crises 


ave rage . 
monitoring 
or su rpr i se 


5) Establish a responsive contractor management 
scheme. It is hard for contractors to understand deeply 
he desiderata that obtain for a system like SAFE for 
integration with the most profound processes of 
intelligence production. 


On a project as crucial as SAFE, 

»bove, coupled with nu mber 3 . 

technological competence are probably 


we recommen d number 1 
Deputies with ”TulT 
essential, if only to 
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provide full-time support with undivided loyalties, 
must, of course, have the rank and authority to deal 
their contractor counterparts. 


They 
w i t h 


als ° that, some outside expert advice would 
be helpful; and for 3) above, a standing Advisory Council on 
Technology (ACT) would seem most fitting. It is not "clear 
whether the council should restrict its considerations 'to 
S f E ' should in fact ultimately deal with a broader range 
of technological problems. These questions are amplified 
below in Section 6. H uiea 


3. A True Pilot SAFE 


Interim SAFE was initiated in 1974 as a set of 
capabilities on a 370/158 run by OOP. Four main capabilities 
were sought for the original SAFE project, and they are 
still valid today: 


1) A mail/message/distribution system 

2) Private files available on-line for analysts 

3) Public files available on-line for analysts 

4) On-line facilities for read, edit, write, and 
document production 


.These four have to be somewhat extended and modified in 
detail to match either today's purported goals or the real 
needs that underlie the requirements for the system. 

The chief uses made of interim SAFE were: 

1) To provide limited experience for certain 
analysts in order to survey their expressed needs. 
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2) To 
echelons, 
processes . 


. demonstrate the capabilities to management 
in order to help with the budget and funding 


3) To derive 
serve as guides 
specifications. 


certain specifications that 
for the actual SAFE 


might 

system 


4) To illustrate 
intelligence environment 
contractors. 


the capabilities in the 
to possible proposed SAFE 


It is important to observe 
used as a pilot sys t e m a : 


t h a t Interim SAFE was never 
-that term.. is 'used" in 


f$~st e m whnlr ha F 1S ' t0 . provide experience with a small 
what the ^ Performance is operationally projected to be 

the n;i n ! ® YStem ° Ught to . be ' In practice, of course, 

P lot system serves in engineering to modify 
requirements and specifications in both usage and 
engineering. y ana 


In Interim SAFE, we were 
statistics about usage were not 
be "not representative." 


informed that in general 
gathered because they would 


The questions 
SAFE include: 


that ought to be answered by a true pilot 


1) What are the usage patterns of naive users? 

What are the usage patterns of experienced 


3) What needs for modifications of SAFE 
performance become manifest from the transition of 
naive users to experienced ones? 

4) what are the user documentation and training 
requirements? 

5) What new requirements emerge from the 

experienced usage? 
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6) How should the services provided by SAFE be 
modified to take advantage of new technology?* 

7) what facets of the system can be safely frozen 
in .design concept? What parts must be carefully 
engineered to retain flexibility of function and 
perf o nuance? 

Only a few of these questions can be dealt with through 
the current Interim SAFE. But there has been a great deal of 
experience elsewhere in systems similar in nature and size 
to SAFE; we note the unique aspects of much of the 
intelligence environment, which is why a true pilot SAFE is 
needed. But if advantage is taken of the continuing 
experience of. these other systems, it is likely that the 
process of . i ni t i at i ng a pilot SAFE and interacting with it 
to guide final SAFE development can be speeded up. 

The questions above that seem at first glance to be 
uniquely answerable by pilot SAFE are 1, 2, 3, and 5. These 
have . to do with analyst usage of the tools and capabilities 
provided to them. W e are the refo re suggesting that a first 

collecting systemat i c lonqitudinal da ta on 

J 5.3 ly.g t_u§ gag— £ — tft e curr en t Interim SA FE, continuing "during 
a conversion to a true pilot SAFE. 

There is little doubt that the general capabilities 
sought can be provided on a small scale by any of a large 
number of current i ns ta 1 lat i ons outside the community, as 
well as inside. At least four members of STAP have had wide 
experience with such systems. Such systems also provide some 
of the extra capabilities that are not now planned as part 
of SAFE, but that are considered highly desirable, 
including: 

1) collaborative capabilities, whereby several 
analysts can simultaneously and remotely collaborate on 
the same task. 


* p or example, good split-screen graphics terminals can 
take . advantage of more powerful editing capabilities than 
previous terminals. 
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2) general communications access to remote files 
over community or common carrier lines; this is 
relevant to making SAFE truly a community-wide 
resource, as is being strongly urged by certain members 
of the IC staff. 

3) new evolutionary user languages, including the 
capability for user on-line control of multiple jobs 
simultaneously. 

The actual implementa ti on of pilot SAFE can th ereto r e 

be h and led in se ver a I wa y s : ' 

1) Use the interim SAFE now running in OOP. 
Upgrade its capabilities, installing the above desired 
new capabilities, aiming at an integrated single system 
for continuing development. 

2) Use an existing system from outside, but 
installed at the CIA, using outside contractor support 
as well as in house personnel. 

3) Use an existing system at an outside 
contractor's installation, sending analysts on TOY to 
provide the usage, etc. The difficulties involved in 
this option are obvious, and it is included only for 
the sake of completeness. 

The advantages of the first option are that interim 
SAFE is now running here, that both users and system 
personnel are familiar with it, and that it performs some of 
the needed capabilities already in the desired intelligence 
environment. The disadvantages are that some system 
reprogramming will be necessary to bring it to state of the 
art, and that this will likely need system architecture and 
programming resources beyond what is available here. 

The advantage of the second option is that such a pilot 
system is nearly an off-the-shelf item, and could be running 
in the CIA fairly quickly. On the other hand, the special 
requirements of the community environment would enormously 
delay the user population as well as the systems and 
programming personnel at the CIA. 
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In s u mrn ary, we stron g ly sin: gest option I , i . e, , u se the 
curren t, i nteri m SAFE. A conscientious effort toward the "Find 
of pilot SAFE being discussed here entails: 

1) Increased technological resources available in 
house. This should include some contractor (TRW) 
personnel, plus some non-TRW contractor personnel, to 
keep the former honest, as it were. A total of six 
people for the first eight months would be needed, aru? 
then perhaps dropping to four on a continuing basis. 

2) Enlarging the user population and the user 
studies. This is so important, and has so many 
ramifications, that is the subject of a separate 
Section 4. 

3) Establishing a direct COINS link that will 
enable study and experimentation by analysts in 
community-wide access and retrieval. 

4) Initiating data gathering and analysis of usage 
pattern and changes in usage patterns by analyst users 
(see also Section 5) 

5) Testing prepared user languages and other tools 
as soon as possible. 


4. The Users of SAFE 


There are two main reasons why the population of users 

of Pilot SAFE must be enlarged and modified from that of 

users of Internum SAFE: 

1) We need to find out answers to what users do, 
how they do it, and how they change. 

2) We need users to find out what SAFE can do 

for them, and how it can be responsive to 

their developing needs. 


The first dictates the initiation of the continuing study of 
user patterns of usage that we have already mentioned. An 
IBM study of a somewhat different user community illustrates 
the approaches and the attacks that might be considered; it 
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is included here as Appendix II. Beyond that, there are a 
number of points that are very hard to pin d^wn, for they 
eal with finding out the unexpected: What new requirements 

-vitem U has%ei - hey become experienced? Every large 
system has felt the impact of such phenomena. Indeed, part 

of the responsibility of management should be to make sure 

dimlv fe't in o n h GdS ' GVe .' U they are P° orl y expressed or 
dimly felt, can be recognized, evaluated, and fulfilled. 

In the beginnings of large scale computer utilization 
computer users submitted jobs to the computer center, which 

c"vs° n ?he r tPUt \ USUaUy Within h °" rS ' a " d sometimes 

days. The case for interactive use of computers was made 

very strongly in the 60s. It.was mistitled time-sharing, but 

t )e lm P ac t was the rapid response and control that it 

provided the user. What it means for the Intelligence 

Community is that the analyst can not only get output fast 

but can also guide the processing and retrievals so as to 

m!' 1 X requ i remen ts that may be hard to specify ahead of 

time. Many computer programmers, especially beginners, enjoy 

the actual process of interaction with a responsive 

computer; but in fact the interaction is valuable only as it 

cu„s down the amount of time needed to specify whathas to 

be done--that is to out down on the amount of interaction 

itoelf. How to do that depends very largely indeed on the 

n l 1 L an h? nUiVe Styles ° f the computer users? the 

their' n, ! ° ne reason wh V study of them and 

h lr practices will pay such big dividends. 

_ lhe . f econd need is of course at least partly 
bureaucratic. The users in the community are busy in the 
ways that they have diligently discovered serve them and 
their task the best. If we expect them to change their ways, 
we have to persuade them that it will be in their best, 
self-interest to do so. To do that, we must make sure that 
there are visible and successful new users in the fields 
that we hope to serve. It is not useful to tell a political 
analyst that it will pay him to use a computer by showinq 
him how a weapons analyst can deal with signal processing of 
radar data links. The user population should be carefully 
expanded, with users who will help, both in providing 
convincing evidence about the value of SAFE and also in 
participating in the development of the capabilities it can 
give. It is also necessary to pick important problems to 
solve using SAFE. F 
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5. The Methodology of Evaluation 


W , iU b f clear from all the above that there is a 
to behave As hJf be know a bout specifying how we want SAFE 

analysts do now^/but 'hoVthey interactively Till ^ 

during and after substantial experience w i?h it? Y 

reccM^' analyze^ pat^nVc^ ?' ' 

r—H-r-r— intelligence analysis and the HbehavTor "~'n? 

intelligence analysts tk= , rr~ — r~ — - — — -Hglli Lvio r ojt 

conceptual model of how an construct a 

problems = o , , Q t i n analyst reasons about his 

problems, as well as merely how he gathers the 
discussed in earlier serMrmc CA p,-, , c statistics 

without it b k L Can be rnade operational 

• UCh "«n9. ' ■ Nor" can 


methodology. 


One of the underlying drives that leads 
°t a need is that of being able to 
effectiveness of some change in organization 
P*™ 1 - « ut ' f *£ .« h«e only a ^oad^d 
estimate of effectiveness, it is difficult 
whether a change in that estimate is due 
decisions about organization and so on, or to 
tides of the world crises. 


to expression 
evaluate the 
procedure, or 
unstructured 
to determine 
to previous 
the changing 


SAFE 3o?ks 0m but ft Vl^ e ih r t? be ablS t0 tGU more than that 
u ^ ' out also th at it is cost-effective to such and 

such a degree, and in such and such respects. If we do not 
know that, we will not h P ♦. . ao noz 

efficiently to correct errors , asSlgn resources 

rsnahnuiL d e r 1 o r s , add improvements and 

capabilities, and modify performance to meet chanqina 
requirements. u ranging 



discus-ed^i f 01 ?n ^ be ^ n " 1ade rnany times already; STAP has 
aitho?s (or-) Previous report, and one of the present 
tn nnn- I St ™ ssed r lt 4 years ago in a contract report 
M , s . Center. cor the Development of Analytical 
Methodologies (CDAM) ^uai/nicai 
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So we need to know what analysts do as part of the 
intelligence process - the fundamental part, for it is the 
gem for which all the rest of the intelligence handling is 
but the setting and the supporting framework. The critical 
problem of evaluation is to determine and quantify the 
contributions to the overall system operation of every piece 
of the operating hierarchy. 

This galaxy of problems - included in the term 
Methodology of Evaluation - has widespread ramifications. 
Besides needing to know whether a modification of SAFE is 
cost-effective or not, we need to know how to produce 
requirements for training SAFE users. Indeed, this is but 
part of a larger requirement for being able to evaluate 
training for the community as a whole. 

The political and social complications from such 
studies should not be neglected, but they are certainly not 
reasons for delaying them. We believe that most analyst.s 
would welcome study of their behavior in producing 
intelligence analysis. Similarly, it is important, as we 
have emphasized in the Analytical Methodologies report, to 
be sensitive to the efficiency of current analytic practices 
in their current context, so that we can properly aid in the 
transitions to a more computerized context. 

The logical base for these studies is ORD's Analytical 
Methodology Research Division. Some of these studies can be 
carried out totally in— house, but some will probably need 
some contracted assistance. The studies nee_ded can be 
summarized: " 

1) Pilot SAFE usage studies : some questions were 
listed’ in Section 3~] They should be extended 
wi th : 

a) Error and complaint log, with respect to 
system failures, new capabilities, 
inadequacies, documentation, etc. 

b) Special examination of usage by SAFE 
analysts of systems and files outside 
the CIA. 
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2) Other Human Factor Studies 

a) Analyst access to non-computer i zed files 

b) Analyst-analyst; interaction in the 
production of intelligence output 

c) The types and efficiencies of motivation 
for intelligence analysts 

d} Models of intelligence analysts and the 

analytic process 

3 ) f Ev a luation techniques developed for application 
to [as we discussed in the STAP Analytical 
Methodologies report.]: 

a) Intelligence products 

b) Intelligence analysts 

c) Intelligence analysis 

6. An Advisory Council on Technology 


We have suggested that project SAFE management could 
benefit from an Advisory Council on Technology (ACT). Here 
we discuss the detailed duties and responsibilities of such 
a council, who might be the personnel and how it might be 
operated. We defer a suggestion that the council deal with 
the larger problems of technology in the entire community: 
because, first, SAFE is difficult enough; second, it will 
serve a community function, even though just the CIA and the 
DIA are involved initially; and third, such a responsibility 
can be more clearly defined after the ACT has proved its 
worth with SAFE. 

The ACT would be a continuing body, comprising both 
community and outside personnel. The primary duty would be 
to assist the SAFE management. In detail, where so tasked, 
ACT should: 
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1)^ Make recommendations about 
concerning technology, new directions 
interactions. 


major decisions 
and user-system 


2) Make evaluations of past decisions as to their 
t ; ect ° n &AFK operations, intelligence production, and 
future developments in technological applications. 

Irovide cost-benefit analyses for proposed 
changes or extensions to the SAFE capabilities. 

4) Monitor certain variables, like interagency 

usage, or contractor performance on particular 

subtasks. 


5) Provide a continuing link with the 
technological state-of-the-art. 

6) Act as a sounding board, to responsively assist 
the SAFE management in creating long-range future 
pl ans, 


Suc ^ f unc tions cannot be satisfactorily performed by a 
group meeting quarterly. It is the monitoring functions that 
are crucial m demanding quick response from the group — how 
should they^ alert the management that coordination or 
redirection is needed quickly? 


™ that SAFE, or a system with SAFE likp 

capabiliti es, is such a centra l pa r t J o f the f utu re 

^ gene e ce s s i n g. of thi s country, that it i s vital 

— - ALf Jlilig manage ment to represent the best that the 

and the tec hnol ogy can affo rd . i rTthe" crucTal 

transitions and developments that lie immediately ahead 
continuity should also be maintained with the cognizant 
supervisory and review groups. Furthermore, any group like 
our suggested ACT must be perceived as working in support of 
the people who have the in-house responsibilities for SAFE. 


In view of the current needs, the ACT should be prepared to 
exercise both review and monitoring functions with some 
intensity for the next few months especially. The most 
immediate duty must be to advise the DCI and SAFE management 
about the design review and possible acceptance this summer. 

As you know, w§_are o f the opinion that t his ought to be 

— — — ---- i f i ed fi arid it is vi tal t h at_the community 
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understand and support 

order to do that efficiently, the ACT _ ThouTd“TnTeg rate the 
ictlons bv SAFE nprsnnnol = , 


FI 1 e . d e c i s i o ns th a t w i 1 _1 be made. 


; . ■>- XV..J.CU i_xy , Lue air snouid integrate the 

current projections by SAFE personnel about analyst usage in 
the future with the experience gained by outside efforts, as 
discussed in previous sections. 

® t hat the ACT review some of the 

ggHL E? , r ab Ig-Qutside s ys te rns mentio ne d a bo ve , " t o a e t"h eT““w”f Fh 

p_riate. ger sonne l_from S AFE its e 1 f , s u ch rev i ews“shou 1 d 

minim, !m CU f 3 hh Y; ^ W ° Uld hope that the c ? rou P Could spend a 
workTnn V* 6 dayS at ea ch installation, and possibly a 
working week, so as to receive not only system 
understanding, but also detailed user experience. It is 
important also for the group to receive views about the 
hiotory and development of each system, in order to be able 
to apply the lessons learned to the community task. 
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